Memory controllers field and execute memory access requests, for example requests to read data from, and write data to, a plurality of memory modules. A memory access request may be initiated by either a central processing unit (CPU) or an input/output (I/O) device.
Computers require fast access to portions of computer memory to enable timely execution of instructions that are stored in the memory and are subsequently executed by the computer processor. Memory caches may be provided on a processor, or in nearby proximity. Recently-accessed data and/or pre-fetched data are held in the cache along with the associated main memory address. Memory access requests are first evaluated against the data held in the memory cache. If the requested information is cached, the information is retrieved therefrom and the main memory access request is terminated. A common cache performance metric is referred to as cache hit rate—the percentage of memory access requests satisfied by the cache. Latency caused by cache miss—memory access requests not satisfied by the cache—is a performance problem in the execution of computer-based instructions.
Typically the speed of operation of the processor is faster than the speed of access to cache memory. A cache hit refers to a processor accessing information in the cache. When the processor is not able to access information in the cache this is referred to herein as a “cache miss.” Cache miss latency has increased as the disparity between the speed required for processor operations and the speed required to access the memory has increased.
Pre-fetching is the fetching of instructions into the cache before they are requested. Pre-fetching information speeds up processing time because the processor can access data in the cache in less time than that required to retrieve the same information from main memory. Pre-fetching of information that is not ultimately requested or that is requested after the pre-fetched information has been displaced from the cache may be detrimental to system performance and unnecessarily increases latency. Generating timely pre-fetches has been a problem with conventional pre-fetching solutions.
A pre-fetch is useless if it brings a line into the cache which will not be used before it is displaced from the cache. Moreover, performing a pre-fetch that is ultimately displaced is counterproductive in that the pre-fetch operation consumes system bandwidth that could have been otherwise used for retrieving requested instructions or data.
A problem with pre-fetching is obtaining the appropriate coverage of a pre-fetch. It will be appreciated that coverage is the identification of useful pre-fetched instruction requests while minimizing useless pre-fetched instruction requests. Attempting to obtain optimal coverage can increase the probability of useless pre-fetches. That is, a more liberal issuance of pre-fetches may increase the probability of useless pre-fetches.